Data Description¶
The data provided is a transformed version of the original data which was collected using sensors.
- Train.csv - To be used for training and tuning of models.
- Test.csv - To be used only for testing the performance of the final best model.
Both the datasets consist of 40 predictor variables and 1 target variable.
Installing and Importing the necessary libraries¶
# Installing the libraries with the specified version
#!pip install --no-deps tensorflow==2.18.0 scikit-learn==1.3.2 matplotlib===3.8.3 seaborn==0.13.2 numpy==1.26.4 pandas==2.2.2 -q --user --no-warn-script-location
#Library for data manupulation and analysis
import numpy as np
#Package for scientific computation
import pandas as pd
#Library for data visualization
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import confusion_matrix
#Splitting dataset in training and test
from sklearn.model_selection import train_test_split
#Time function
import time
#import tools for data processing
from sklearn.preprocessing import StandardScaler,LabelEncoder,OneHotEncoder
#import simplerimputer
from sklearn.impute import SimpleImputer
#Import function for evaluating performance of model
from sklearn.metrics import f1_score,accuracy_score,precision_score,recall_score,classification_report
#import metrics
from sklearn import metrics
#Imports the tensorflow,keras and layers
import tensorflow
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential # Corrected import statement
from tensorflow.keras.layers import Dense,Dropout,Input,BatchNormalization
from tensorflow.keras.optimizers import Adam,SGD
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.utils import to_categorical
from tensorflow.keras import backend
# to suppress unnecessary warnings
import warnings
warnings.filterwarnings("ignore")
# Set the seed using keras.utils.set_random_seed. This will set:
# 1) `numpy` seed
# 2) backend random seed
# 3) `python` random seed
keras.utils.set_random_seed(812)
# If using TensorFlow, this will make GPU ops as deterministic as possible,
# but it might affect the overall performance
tf.config.experimental.enable_op_determinism()
Loading the Data¶
# uncomment and run the following lines for Google Colab
#from google.colab import drive
#drive.mount('/content/drive')
data=pd.read_csv('Train.csv')
data_test=pd.read_csv('Test.csv')
#creating copies
df=data.copy()
df_test=data_test.copy()
Data Overview¶
Understand shape of the dataset¶
#shape of the Train dataset
print(f'The Train dataset has {df.shape[0]} rows and {df.shape[1]} columns ')
The Train dataset has 20000 rows and 41 columns
#shape of the Test dataset
print(f'The Test dataset has {df_test.shape[0]} rows and {df_test.shape[1]} columns ')
The Test dataset has 5000 rows and 41 columns
View the first 5 rows of the datasets¶
#First five raws of train
df.head()
| V1 | V2 | V3 | V4 | V5 | V6 | V7 | V8 | V9 | V10 | ... | V32 | V33 | V34 | V35 | V36 | V37 | V38 | V39 | V40 | Target | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -4.464606 | -4.679129 | 3.101546 | 0.506130 | -0.221083 | -2.032511 | -2.910870 | 0.050714 | -1.522351 | 3.761892 | ... | 3.059700 | -1.690440 | 2.846296 | 2.235198 | 6.667486 | 0.443809 | -2.369169 | 2.950578 | -3.480324 | 0 |
| 1 | 3.365912 | 3.653381 | 0.909671 | -1.367528 | 0.332016 | 2.358938 | 0.732600 | -4.332135 | 0.565695 | -0.101080 | ... | -1.795474 | 3.032780 | -2.467514 | 1.894599 | -2.297780 | -1.731048 | 5.908837 | -0.386345 | 0.616242 | 0 |
| 2 | -3.831843 | -5.824444 | 0.634031 | -2.418815 | -1.773827 | 1.016824 | -2.098941 | -3.173204 | -2.081860 | 5.392621 | ... | -0.257101 | 0.803550 | 4.086219 | 2.292138 | 5.360850 | 0.351993 | 2.940021 | 3.839160 | -4.309402 | 0 |
| 3 | 1.618098 | 1.888342 | 7.046143 | -1.147285 | 0.083080 | -1.529780 | 0.207309 | -2.493629 | 0.344926 | 2.118578 | ... | -3.584425 | -2.577474 | 1.363769 | 0.622714 | 5.550100 | -1.526796 | 0.138853 | 3.101430 | -1.277378 | 0 |
| 4 | -0.111440 | 3.872488 | -3.758361 | -2.982897 | 3.792714 | 0.544960 | 0.205433 | 4.848994 | -1.854920 | -6.220023 | ... | 8.265896 | 6.629213 | -10.068689 | 1.222987 | -3.229763 | 1.686909 | -2.163896 | -3.644622 | 6.510338 | 0 |
5 rows × 41 columns
#First five raws of test
df_test.head()
| V1 | V2 | V3 | V4 | V5 | V6 | V7 | V8 | V9 | V10 | ... | V32 | V33 | V34 | V35 | V36 | V37 | V38 | V39 | V40 | Target | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -0.613489 | -3.819640 | 2.202302 | 1.300420 | -1.184929 | -4.495964 | -1.835817 | 4.722989 | 1.206140 | -0.341909 | ... | 2.291204 | -5.411388 | 0.870073 | 0.574479 | 4.157191 | 1.428093 | -10.511342 | 0.454664 | -1.448363 | 0 |
| 1 | 0.389608 | -0.512341 | 0.527053 | -2.576776 | -1.016766 | 2.235112 | -0.441301 | -4.405744 | -0.332869 | 1.966794 | ... | -2.474936 | 2.493582 | 0.315165 | 2.059288 | 0.683859 | -0.485452 | 5.128350 | 1.720744 | -1.488235 | 0 |
| 2 | -0.874861 | -0.640632 | 4.084202 | -1.590454 | 0.525855 | -1.957592 | -0.695367 | 1.347309 | -1.732348 | 0.466500 | ... | -1.318888 | -2.997464 | 0.459664 | 0.619774 | 5.631504 | 1.323512 | -1.752154 | 1.808302 | 1.675748 | 0 |
| 3 | 0.238384 | 1.458607 | 4.014528 | 2.534478 | 1.196987 | -3.117330 | -0.924035 | 0.269493 | 1.322436 | 0.702345 | ... | 3.517918 | -3.074085 | -0.284220 | 0.954576 | 3.029331 | -1.367198 | -3.412140 | 0.906000 | -2.450889 | 0 |
| 4 | 5.828225 | 2.768260 | -1.234530 | 2.809264 | -1.641648 | -1.406698 | 0.568643 | 0.965043 | 1.918379 | -2.774855 | ... | 1.773841 | -1.501573 | -2.226702 | 4.776830 | -6.559698 | -0.805551 | -0.276007 | -3.858207 | -0.537694 | 0 |
5 rows × 41 columns
check the datatypes of column of the datasets¶
#datatype for train
df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 20000 entries, 0 to 19999 Data columns (total 41 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 V1 19982 non-null float64 1 V2 19982 non-null float64 2 V3 20000 non-null float64 3 V4 20000 non-null float64 4 V5 20000 non-null float64 5 V6 20000 non-null float64 6 V7 20000 non-null float64 7 V8 20000 non-null float64 8 V9 20000 non-null float64 9 V10 20000 non-null float64 10 V11 20000 non-null float64 11 V12 20000 non-null float64 12 V13 20000 non-null float64 13 V14 20000 non-null float64 14 V15 20000 non-null float64 15 V16 20000 non-null float64 16 V17 20000 non-null float64 17 V18 20000 non-null float64 18 V19 20000 non-null float64 19 V20 20000 non-null float64 20 V21 20000 non-null float64 21 V22 20000 non-null float64 22 V23 20000 non-null float64 23 V24 20000 non-null float64 24 V25 20000 non-null float64 25 V26 20000 non-null float64 26 V27 20000 non-null float64 27 V28 20000 non-null float64 28 V29 20000 non-null float64 29 V30 20000 non-null float64 30 V31 20000 non-null float64 31 V32 20000 non-null float64 32 V33 20000 non-null float64 33 V34 20000 non-null float64 34 V35 20000 non-null float64 35 V36 20000 non-null float64 36 V37 20000 non-null float64 37 V38 20000 non-null float64 38 V39 20000 non-null float64 39 V40 20000 non-null float64 40 Target 20000 non-null int64 dtypes: float64(40), int64(1) memory usage: 6.3 MB
#datatype for test
df_test.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 5000 entries, 0 to 4999 Data columns (total 41 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 V1 4995 non-null float64 1 V2 4994 non-null float64 2 V3 5000 non-null float64 3 V4 5000 non-null float64 4 V5 5000 non-null float64 5 V6 5000 non-null float64 6 V7 5000 non-null float64 7 V8 5000 non-null float64 8 V9 5000 non-null float64 9 V10 5000 non-null float64 10 V11 5000 non-null float64 11 V12 5000 non-null float64 12 V13 5000 non-null float64 13 V14 5000 non-null float64 14 V15 5000 non-null float64 15 V16 5000 non-null float64 16 V17 5000 non-null float64 17 V18 5000 non-null float64 18 V19 5000 non-null float64 19 V20 5000 non-null float64 20 V21 5000 non-null float64 21 V22 5000 non-null float64 22 V23 5000 non-null float64 23 V24 5000 non-null float64 24 V25 5000 non-null float64 25 V26 5000 non-null float64 26 V27 5000 non-null float64 27 V28 5000 non-null float64 28 V29 5000 non-null float64 29 V30 5000 non-null float64 30 V31 5000 non-null float64 31 V32 5000 non-null float64 32 V33 5000 non-null float64 33 V34 5000 non-null float64 34 V35 5000 non-null float64 35 V36 5000 non-null float64 36 V37 5000 non-null float64 37 V38 5000 non-null float64 38 V39 5000 non-null float64 39 V40 5000 non-null float64 40 Target 5000 non-null int64 dtypes: float64(40), int64(1) memory usage: 1.6 MB
In both datasets Columns V1 to V40 are float while target is integer and there are missing values in V1 and V2
Checking for missing values¶
#Checking for missing values in train
df.isnull().sum()
| 0 | |
|---|---|
| V1 | 18 |
| V2 | 18 |
| V3 | 0 |
| V4 | 0 |
| V5 | 0 |
| V6 | 0 |
| V7 | 0 |
| V8 | 0 |
| V9 | 0 |
| V10 | 0 |
| V11 | 0 |
| V12 | 0 |
| V13 | 0 |
| V14 | 0 |
| V15 | 0 |
| V16 | 0 |
| V17 | 0 |
| V18 | 0 |
| V19 | 0 |
| V20 | 0 |
| V21 | 0 |
| V22 | 0 |
| V23 | 0 |
| V24 | 0 |
| V25 | 0 |
| V26 | 0 |
| V27 | 0 |
| V28 | 0 |
| V29 | 0 |
| V30 | 0 |
| V31 | 0 |
| V32 | 0 |
| V33 | 0 |
| V34 | 0 |
| V35 | 0 |
| V36 | 0 |
| V37 | 0 |
| V38 | 0 |
| V39 | 0 |
| V40 | 0 |
| Target | 0 |
#Checking for missing values in test
df_test.isnull().sum()
| 0 | |
|---|---|
| V1 | 5 |
| V2 | 6 |
| V3 | 0 |
| V4 | 0 |
| V5 | 0 |
| V6 | 0 |
| V7 | 0 |
| V8 | 0 |
| V9 | 0 |
| V10 | 0 |
| V11 | 0 |
| V12 | 0 |
| V13 | 0 |
| V14 | 0 |
| V15 | 0 |
| V16 | 0 |
| V17 | 0 |
| V18 | 0 |
| V19 | 0 |
| V20 | 0 |
| V21 | 0 |
| V22 | 0 |
| V23 | 0 |
| V24 | 0 |
| V25 | 0 |
| V26 | 0 |
| V27 | 0 |
| V28 | 0 |
| V29 | 0 |
| V30 | 0 |
| V31 | 0 |
| V32 | 0 |
| V33 | 0 |
| V34 | 0 |
| V35 | 0 |
| V36 | 0 |
| V37 | 0 |
| V38 | 0 |
| V39 | 0 |
| V40 | 0 |
| Target | 0 |
There are missing values for V1 and V2 in both datasets
Checking for duplicates¶
#Duplicates in train
df.duplicated().sum()
np.int64(0)
There are no duplicates in the train dataset
Let's check the statistical summary of the data¶
#statistical summary of train
df.describe().T
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| V1 | 19982.0 | -0.271996 | 3.441625 | -11.876451 | -2.737146 | -0.747917 | 1.840112 | 15.493002 |
| V2 | 19982.0 | 0.440430 | 3.150784 | -12.319951 | -1.640674 | 0.471536 | 2.543967 | 13.089269 |
| V3 | 20000.0 | 2.484699 | 3.388963 | -10.708139 | 0.206860 | 2.255786 | 4.566165 | 17.090919 |
| V4 | 20000.0 | -0.083152 | 3.431595 | -15.082052 | -2.347660 | -0.135241 | 2.130615 | 13.236381 |
| V5 | 20000.0 | -0.053752 | 2.104801 | -8.603361 | -1.535607 | -0.101952 | 1.340480 | 8.133797 |
| V6 | 20000.0 | -0.995443 | 2.040970 | -10.227147 | -2.347238 | -1.000515 | 0.380330 | 6.975847 |
| V7 | 20000.0 | -0.879325 | 1.761626 | -7.949681 | -2.030926 | -0.917179 | 0.223695 | 8.006091 |
| V8 | 20000.0 | -0.548195 | 3.295756 | -15.657561 | -2.642665 | -0.389085 | 1.722965 | 11.679495 |
| V9 | 20000.0 | -0.016808 | 2.160568 | -8.596313 | -1.494973 | -0.067597 | 1.409203 | 8.137580 |
| V10 | 20000.0 | -0.012998 | 2.193201 | -9.853957 | -1.411212 | 0.100973 | 1.477045 | 8.108472 |
| V11 | 20000.0 | -1.895393 | 3.124322 | -14.832058 | -3.922404 | -1.921237 | 0.118906 | 11.826433 |
| V12 | 20000.0 | 1.604825 | 2.930454 | -12.948007 | -0.396514 | 1.507841 | 3.571454 | 15.080698 |
| V13 | 20000.0 | 1.580486 | 2.874658 | -13.228247 | -0.223545 | 1.637185 | 3.459886 | 15.419616 |
| V14 | 20000.0 | -0.950632 | 1.789651 | -7.738593 | -2.170741 | -0.957163 | 0.270677 | 5.670664 |
| V15 | 20000.0 | -2.414993 | 3.354974 | -16.416606 | -4.415322 | -2.382617 | -0.359052 | 12.246455 |
| V16 | 20000.0 | -2.925225 | 4.221717 | -20.374158 | -5.634240 | -2.682705 | -0.095046 | 13.583212 |
| V17 | 20000.0 | -0.134261 | 3.345462 | -14.091184 | -2.215611 | -0.014580 | 2.068751 | 16.756432 |
| V18 | 20000.0 | 1.189347 | 2.592276 | -11.643994 | -0.403917 | 0.883398 | 2.571770 | 13.179863 |
| V19 | 20000.0 | 1.181808 | 3.396925 | -13.491784 | -1.050168 | 1.279061 | 3.493299 | 13.237742 |
| V20 | 20000.0 | 0.023608 | 3.669477 | -13.922659 | -2.432953 | 0.033415 | 2.512372 | 16.052339 |
| V21 | 20000.0 | -3.611252 | 3.567690 | -17.956231 | -5.930360 | -3.532888 | -1.265884 | 13.840473 |
| V22 | 20000.0 | 0.951835 | 1.651547 | -10.122095 | -0.118127 | 0.974687 | 2.025594 | 7.409856 |
| V23 | 20000.0 | -0.366116 | 4.031860 | -14.866128 | -3.098756 | -0.262093 | 2.451750 | 14.458734 |
| V24 | 20000.0 | 1.134389 | 3.912069 | -16.387147 | -1.468062 | 0.969048 | 3.545975 | 17.163291 |
| V25 | 20000.0 | -0.002186 | 2.016740 | -8.228266 | -1.365178 | 0.025050 | 1.397112 | 8.223389 |
| V26 | 20000.0 | 1.873785 | 3.435137 | -11.834271 | -0.337863 | 1.950531 | 4.130037 | 16.836410 |
| V27 | 20000.0 | -0.612413 | 4.368847 | -14.904939 | -3.652323 | -0.884894 | 2.189177 | 17.560404 |
| V28 | 20000.0 | -0.883218 | 1.917713 | -9.269489 | -2.171218 | -0.891073 | 0.375884 | 6.527643 |
| V29 | 20000.0 | -0.985625 | 2.684365 | -12.579469 | -2.787443 | -1.176181 | 0.629773 | 10.722055 |
| V30 | 20000.0 | -0.015534 | 3.005258 | -14.796047 | -1.867114 | 0.184346 | 2.036229 | 12.505812 |
| V31 | 20000.0 | 0.486842 | 3.461384 | -13.722760 | -1.817772 | 0.490304 | 2.730688 | 17.255090 |
| V32 | 20000.0 | 0.303799 | 5.500400 | -19.876502 | -3.420469 | 0.052073 | 3.761722 | 23.633187 |
| V33 | 20000.0 | 0.049825 | 3.575285 | -16.898353 | -2.242857 | -0.066249 | 2.255134 | 16.692486 |
| V34 | 20000.0 | -0.462702 | 3.183841 | -17.985094 | -2.136984 | -0.255008 | 1.436935 | 14.358213 |
| V35 | 20000.0 | 2.229620 | 2.937102 | -15.349803 | 0.336191 | 2.098633 | 4.064358 | 15.291065 |
| V36 | 20000.0 | 1.514809 | 3.800860 | -14.833178 | -0.943809 | 1.566526 | 3.983939 | 19.329576 |
| V37 | 20000.0 | 0.011316 | 1.788165 | -5.478350 | -1.255819 | -0.128435 | 1.175533 | 7.467006 |
| V38 | 20000.0 | -0.344025 | 3.948147 | -17.375002 | -2.987638 | -0.316849 | 2.279399 | 15.289923 |
| V39 | 20000.0 | 0.890653 | 1.753054 | -6.438880 | -0.272250 | 0.919261 | 2.057540 | 7.759877 |
| V40 | 20000.0 | -0.875630 | 3.012155 | -11.023935 | -2.940193 | -0.920806 | 1.119897 | 10.654265 |
| Target | 20000.0 | 0.055500 | 0.228959 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.000000 |
Exploratory Data Analysis¶
Univariate analysis¶
def histogram_boxplot(data, feature, figsize=(15, 10), kde=False, bins=None):
"""
Boxplot and histogram combined
data: dataframe
feature: dataframe column
figsize: size of figure (default (15,10))
kde: whether to show the density curve (default False)
bins: number of bins for histogram (default None)
"""
f2, (ax_box2, ax_hist2) = plt.subplots(
nrows=2, # Number of rows of the subplot grid= 2
sharex=True, # x-axis will be shared among all subplots
gridspec_kw={"height_ratios": (0.25, 0.75)},
figsize=figsize,
) # creating the 2 subplots
sns.boxplot(
data=data, x=feature, ax=ax_box2, showmeans=True, color="violet"
) # boxplot will be created and a triangle will indicate the mean value of the column
sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins
) if bins else sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2
) # For histogram
ax_hist2.axvline(
data[feature].mean(), color="green", linestyle="--"
) # Add mean to the histogram
ax_hist2.axvline(
data[feature].median(), color="black", linestyle="-"
) # Add median to the histogram
For Features in Train dataset
cols_list = df.select_dtypes(include=np.number).columns.tolist()
cols_list.remove("Target")
for feature in cols_list:
histogram_boxplot(df, feature)
The features in the dataset V1 to V40 are almost symentrical in distribution.
Distribution of target¶
#Distribution in Train
df.Target.value_counts(normalize=True)
| proportion | |
|---|---|
| Target | |
| 0 | 0.9445 |
| 1 | 0.0555 |
#Distribution in test
df_test.Target.value_counts(normalize=True)
| proportion | |
|---|---|
| Target | |
| 0 | 0.9436 |
| 1 | 0.0564 |
The dataset is imbalance with no failure having significantly fewer instances
Bivariate Analysis¶
#Plot correlation between features
plt.figure(figsize=(30,20))
sns.heatmap(df[cols_list].corr(),annot=True,vmin=-1,vmax=1,cmap='BrBG')
plt.show()
The highest negative correlation is between V14 and V2 at 0.85 while highest positive correlation is between V15 and V7 at 0.87.
Creating functions that will help us with further analysis.¶
### function to plot distributions wrt target
def distribution_plot_wrt_target(data, predictor, target):
fig, axs = plt.subplots(2, 2, figsize=(12, 10))
target_uniq = data[target].unique()
axs[0, 0].set_title("Distribution of target for target=" + str(target_uniq[0]))
sns.histplot(
data=data[data[target] == target_uniq[0]],
x=predictor,
kde=True,
ax=axs[0, 0],
color="teal",
stat="density",
)
axs[0, 1].set_title("Distribution of target for target=" + str(target_uniq[1]))
sns.histplot(
data=data[data[target] == target_uniq[1]],
x=predictor,
kde=True,
ax=axs[0, 1],
color="orange",
stat="density",
)
axs[1, 0].set_title("Boxplot w.r.t target")
sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette="gist_rainbow")
axs[1, 1].set_title("Boxplot (without outliers) w.r.t target")
sns.boxplot(
data=data,
x=target,
y=predictor,
ax=axs[1, 1],
showfliers=False,
palette="gist_rainbow",
)
plt.tight_layout()
plt.show()
for feature in cols_list:
distribution_plot_wrt_target(df, feature, 'Target')
There is no variation in distribution between features V2,V6,V12,V25,V27,V30 and V37 with Target. While the other features display Variation with Target possible indication of impact.
Data Preprocessing¶
Separating response variable and predictors in train dataset¶
X=df.drop('Target',axis=1)
y=df['Target']
#printing the shape of the data
print(X.shape)
print(y.shape)
(20000, 40) (20000,)
Splitting the train dataset into train and Validation dataset¶
X_train,X_val,y_train,y_val=train_test_split(X,y,test_size=0.2,random_state=1,stratify=y)
#checking the shape of dataset
print(X_train.shape)
print(X_val.shape)
print(y_train.shape)
print(y_val.shape)
(16000, 40) (4000, 40) (16000,) (4000,)
Separating response variable and predictors in test dataset¶
X_test=df_test.drop('Target',axis=1)
y_test=df_test['Target']
print(X_test.shape)
print(y_test.shape)
(5000, 40) (5000,)
Imputing missing values¶
V1 and V2 had missing values we will impute the missing values using median.
#defining SimpleImputer
imputer=SimpleImputer(strategy='median')
#Fit and transform training data
X_train=pd.DataFrame(imputer.fit_transform(X_train),columns=X_train.columns)
#Transform validation data
X_val=pd.DataFrame(imputer.transform(X_val),columns=X_train.columns)
#Transform test data
X_test=pd.DataFrame(imputer.transform(X_test),columns=X_train.columns)
#checking missing values
print(X_train.isnull().sum().sum())
print(X_val.isnull().sum().sum())
print(X_test.isnull().sum().sum())
0 0 0
Model Building¶
Model Evaluation Criterion¶
The model will make predictions as follows:
- True positives (TP) are failures correctly predicted by the model. These will result in repairing costs.
- False negatives (FN) are real failures where there is no detection by the model. These will result in replacement costs.
- False positives (FP) are detections where there is no failure. These will result in inspection costs.
Since it is given that the cost of repairing a generator is much less than the cost of replacing it, and the cost of inspection is less than the cost of repair hence,the company might be interested most in minimising cost of replacement by reducing FN which can be achieved by Maximising Recall
As we have are dealing with an imbalance in class distribution, we will be using class weights to allow the model to give proportionally more importance to the minority class.
# Calculate class weights for imbalanced dataset
cw = (y_train.shape[0]) / np.bincount(y_train)
# Create a dictionary mapping class indices to their respective class weights
cw_dict = {}
for i in range(cw.shape[0]):
cw_dict[i] = cw[i]
cw_dict
{0: np.float64(1.0587612493382743), 1: np.float64(18.01801801801802)}
Utility functions¶
def plot(history, name):
"""
Function to plot loss/accuracy
history: an object which stores the metrics and losses.
name: can be one of Loss or Accuracy
"""
#Creating a subplot with figure and axes.
fig, ax = plt.subplots()
#Plotting the train accuracy or train loss
plt.plot(history.history[name])
#Plotting the validation accuracy or validation loss
plt.plot(history.history['val_'+name])
#Defining the title of the plot.
plt.title('Model ' + name.capitalize())
#Capitalizing the first letter.
plt.ylabel(name.capitalize())
#Defining the label for the x-axis.
plt.xlabel('Epoch')
#Defining the legend, loc controls the position of the legend.
fig.legend(['Train', 'Validation'], loc="outside right upper")
dataframe to store the results from all the models we build¶
# defining a function to compute different metrics to check performance of a classification model built using statsmodels
def model_performance_classification(
model, predictors, target, threshold=0.5
):
"""
Function to compute different metrics to check classification model performance
model: classifier
predictors: independent variables
target: dependent variable
threshold: threshold for classifying the observation as class 1
"""
# checking which probabilities are greater than threshold
pred = model.predict(predictors) > threshold
# to compute Accuracy
acc = accuracy_score(target, pred)
# to compute Recall
recall = recall_score(target, pred, average='weighted')
# to compute Precision
precision = precision_score(target, pred, average='weighted')
# to compute F1-score
f1 = f1_score(target, pred, average='weighted')
# creating a dataframe of metrics
df_perf = pd.DataFrame(
{"Accuracy": acc, "Recall": recall, "Precision": precision, "F1 Score": f1,},
index=[0],
)
return df_perf
Initial Model Building (Model 0)¶
- Let's start with a neural network consisting of
- just one hidden layer
- activation function of ReLU
- SGD as the optimizer
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_0 = Sequential()
model_0.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_0.add(Dense(32,activation="relu"))
model_0.add(Dense(1,activation = 'sigmoid'))
#compiling the model With SDG as optimiser
optimizer=keras.optimizers.SGD()
model_0.compile(optimizer = optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_0.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.7752 - recall: 0.8306 - val_loss: 0.1977 - val_recall: 0.8784 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.4029 - recall: 0.9043 - val_loss: 0.1678 - val_recall: 0.8874 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3533 - recall: 0.9128 - val_loss: 0.1621 - val_recall: 0.8874 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3282 - recall: 0.9154 - val_loss: 0.1571 - val_recall: 0.8964 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3092 - recall: 0.9151 - val_loss: 0.1509 - val_recall: 0.8964 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2935 - recall: 0.9169 - val_loss: 0.1526 - val_recall: 0.8964 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2805 - recall: 0.9229 - val_loss: 0.1490 - val_recall: 0.9009 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2688 - recall: 0.9239 - val_loss: 0.1413 - val_recall: 0.8964 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2600 - recall: 0.9268 - val_loss: 0.1406 - val_recall: 0.8964 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2516 - recall: 0.9235 - val_loss: 0.1320 - val_recall: 0.8919 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2394 - recall: 0.9283 - val_loss: 0.1305 - val_recall: 0.8919 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2364 - recall: 0.9276 - val_loss: 0.1269 - val_recall: 0.8964 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2307 - recall: 0.9262 - val_loss: 0.1292 - val_recall: 0.8964 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2222 - recall: 0.9312 - val_loss: 0.1287 - val_recall: 0.8964 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2169 - recall: 0.9345 - val_loss: 0.1241 - val_recall: 0.8964 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2113 - recall: 0.9346 - val_loss: 0.1278 - val_recall: 0.8964 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2050 - recall: 0.9321 - val_loss: 0.1218 - val_recall: 0.8964 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1989 - recall: 0.9364 - val_loss: 0.1162 - val_recall: 0.8919 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1991 - recall: 0.9341 - val_loss: 0.1185 - val_recall: 0.8919 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1910 - recall: 0.9348 - val_loss: 0.1173 - val_recall: 0.8919 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1849 - recall: 0.9361 - val_loss: 0.1266 - val_recall: 0.8919 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1792 - recall: 0.9354 - val_loss: 0.1149 - val_recall: 0.8919 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1748 - recall: 0.9387 - val_loss: 0.1168 - val_recall: 0.8874 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1777 - recall: 0.9390 - val_loss: 0.1129 - val_recall: 0.8874 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.1718 - recall: 0.9430 - val_loss: 0.1184 - val_recall: 0.8964 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1693 - recall: 0.9380 - val_loss: 0.1212 - val_recall: 0.8919 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1673 - recall: 0.9401 - val_loss: 0.1171 - val_recall: 0.8829 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1642 - recall: 0.9462 - val_loss: 0.1193 - val_recall: 0.8874 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1463 - recall: 0.9449 - val_loss: 0.1223 - val_recall: 0.8874 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1685 - recall: 0.9389 - val_loss: 0.1120 - val_recall: 0.8874
print("Time taken in seconds ",end-start)
Time taken in seconds 32.86089372634888
model_0.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 4,739 (18.52 KB)
Trainable params: 4,737 (18.50 KB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 2 (12.00 B)
plot(history,"loss")
model_0_train_perf=model_performance_classification(model_0,X_train,y_train)
model_0_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 709us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.98175 | 0.98175 | 0.984608 | 0.982628 |
model_0_val_perf=model_performance_classification(model_0,X_val,y_val)
model_0_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.978 | 0.978 | 0.980238 | 0.978812 |
Both recall for test and train are around 98% percent though wide gap between validation loss function not smooth but oscilating.
Model Performance Improvement¶
Model 1¶
In model 1 we add amomentum to check whether it's accelerating the learning process
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_1 = Sequential()
model_1.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_1.add(Dense(32,activation="relu"))
model_1.add(Dense(1,activation = 'sigmoid'))
#compiling the model With SDG with momentum as optimiser
optimizer =keras.optimizers.SGD(momentum=0.9)
model_1.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_1.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.7851 - recall: 0.8329 - val_loss: 0.2611 - val_recall: 0.8919 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.3576 - recall: 0.9007 - val_loss: 0.2256 - val_recall: 0.9054 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3194 - recall: 0.9109 - val_loss: 0.1662 - val_recall: 0.8964 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3158 - recall: 0.9217 - val_loss: 0.1611 - val_recall: 0.8829 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2944 - recall: 0.9186 - val_loss: 0.1425 - val_recall: 0.8694 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2862 - recall: 0.9219 - val_loss: 0.1462 - val_recall: 0.8874 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2694 - recall: 0.9239 - val_loss: 0.1583 - val_recall: 0.8919 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2779 - recall: 0.9220 - val_loss: 0.1526 - val_recall: 0.8919 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2605 - recall: 0.9287 - val_loss: 0.1419 - val_recall: 0.8784 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2522 - recall: 0.9317 - val_loss: 0.1548 - val_recall: 0.8829 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2517 - recall: 0.9310 - val_loss: 0.1554 - val_recall: 0.8784 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2498 - recall: 0.9317 - val_loss: 0.1703 - val_recall: 0.8829 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2430 - recall: 0.9317 - val_loss: 0.1766 - val_recall: 0.8964 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2397 - recall: 0.9350 - val_loss: 0.1976 - val_recall: 0.8919 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.2573 - recall: 0.9277 - val_loss: 0.1822 - val_recall: 0.8964 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2624 - recall: 0.9342 - val_loss: 0.3325 - val_recall: 0.8919 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2957 - recall: 0.9233 - val_loss: 0.1703 - val_recall: 0.8874 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2483 - recall: 0.9317 - val_loss: 0.2299 - val_recall: 0.8829 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2435 - recall: 0.9343 - val_loss: 0.2095 - val_recall: 0.8784 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2363 - recall: 0.9364 - val_loss: 0.2124 - val_recall: 0.8829 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2370 - recall: 0.9364 - val_loss: 0.2471 - val_recall: 0.8874 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2423 - recall: 0.9373 - val_loss: 0.2205 - val_recall: 0.8874 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2358 - recall: 0.9371 - val_loss: 0.2316 - val_recall: 0.8784 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2812 - recall: 0.9287 - val_loss: 0.2032 - val_recall: 0.8919 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.2538 - recall: 0.9334 - val_loss: 0.2751 - val_recall: 0.8829 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2772 - recall: 0.9282 - val_loss: 0.2316 - val_recall: 0.8919 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2665 - recall: 0.9299 - val_loss: 0.1797 - val_recall: 0.8874 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2458 - recall: 0.9328 - val_loss: 0.2056 - val_recall: 0.8874 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2407 - recall: 0.9363 - val_loss: 0.1574 - val_recall: 0.8784 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2325 - recall: 0.9370 - val_loss: 0.1633 - val_recall: 0.8874
print("Time taken in seconds ",end-start)
Time taken in seconds 37.33909344673157
model_1.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 9,476 (37.02 KB)
Trainable params: 4,737 (18.50 KB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 4,739 (18.52 KB)
plot(history,"loss")
model_1_train_perf=model_performance_classification(model_1,X_train,y_train)
model_1_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 675us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.993437 | 0.993437 | 0.993372 | 0.993393 |
model_1_val_perf=model_performance_classification(model_1,X_val,y_val)
model_1_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 745us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.988 | 0.988 | 0.987951 | 0.987974 |
Both recall in test and train have improved to 99%. The loss plots are non-smooth and oscilating.
Model 2¶
Using Adam optimizer with no regularization
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_2 = Sequential()
model_2.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_2.add(Dense(32,activation="relu"))
model_2.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam()
model_2.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_2.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.8588 - recall: 0.7826 - val_loss: 0.2708 - val_recall: 0.9054 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3898 - recall: 0.9018 - val_loss: 0.2371 - val_recall: 0.9054 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3387 - recall: 0.9118 - val_loss: 0.2323 - val_recall: 0.9009 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3120 - recall: 0.9170 - val_loss: 0.2348 - val_recall: 0.8964 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2956 - recall: 0.9198 - val_loss: 0.2017 - val_recall: 0.8964 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2782 - recall: 0.9208 - val_loss: 0.1886 - val_recall: 0.9009 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2609 - recall: 0.9224 - val_loss: 0.1766 - val_recall: 0.8964 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2499 - recall: 0.9265 - val_loss: 0.1485 - val_recall: 0.8919 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.2417 - recall: 0.9298 - val_loss: 0.1586 - val_recall: 0.8919 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2285 - recall: 0.9304 - val_loss: 0.1449 - val_recall: 0.8829 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2239 - recall: 0.9336 - val_loss: 0.1443 - val_recall: 0.8919 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2147 - recall: 0.9332 - val_loss: 0.1328 - val_recall: 0.8874 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2037 - recall: 0.9355 - val_loss: 0.1361 - val_recall: 0.8874 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1972 - recall: 0.9375 - val_loss: 0.1543 - val_recall: 0.8919 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1915 - recall: 0.9377 - val_loss: 0.1375 - val_recall: 0.8829 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1841 - recall: 0.9394 - val_loss: 0.1317 - val_recall: 0.8784 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1797 - recall: 0.9436 - val_loss: 0.1608 - val_recall: 0.8919 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1746 - recall: 0.9492 - val_loss: 0.1275 - val_recall: 0.8784 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.1639 - recall: 0.9464 - val_loss: 0.1397 - val_recall: 0.8874 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1587 - recall: 0.9514 - val_loss: 0.1407 - val_recall: 0.8829 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1495 - recall: 0.9498 - val_loss: 0.1272 - val_recall: 0.8784 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1436 - recall: 0.9519 - val_loss: 0.1369 - val_recall: 0.8784 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1457 - recall: 0.9530 - val_loss: 0.1292 - val_recall: 0.8829 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1335 - recall: 0.9560 - val_loss: 0.1585 - val_recall: 0.8874 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1410 - recall: 0.9564 - val_loss: 0.1296 - val_recall: 0.8739 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1282 - recall: 0.9552 - val_loss: 0.1201 - val_recall: 0.8784 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1135 - recall: 0.9624 - val_loss: 0.1183 - val_recall: 0.8784 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1143 - recall: 0.9609 - val_loss: 0.1246 - val_recall: 0.8784 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.1140 - recall: 0.9599 - val_loss: 0.1302 - val_recall: 0.8784 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1084 - recall: 0.9633 - val_loss: 0.1190 - val_recall: 0.8784
print("Time taken in seconds ",end-start)
Time taken in seconds 33.61634564399719
model_2.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 14,213 (55.52 KB)
Trainable params: 4,737 (18.50 KB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 9,476 (37.02 KB)
plot(history,"loss")
model_2_train_perf=model_performance_classification(model_2,X_train,y_train)
model_2_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 705us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.987812 | 0.987812 | 0.989453 | 0.988276 |
model_2_val_perf=model_performance_classification(model_2,X_val,y_val)
model_2_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.97925 | 0.97925 | 0.980815 | 0.979846 |
Model perfomance in terms of recall has reduced slightly in both cases though the loss plot are smoothening with slight oscilation
Model 3¶
using Adam optimizer with dropout regularization
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_3 = Sequential()
model_3.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_3.add(Dropout(0.4))
model_3.add(Dense(32,activation="relu"))
model_3.add(Dropout(0.2))
model_3.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam optimiser
optimizer = keras.optimizers.Adam()
model_3.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_3.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.3322 - recall: 0.6637 - val_loss: 0.3584 - val_recall: 0.8919 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6297 - recall: 0.8494 - val_loss: 0.3284 - val_recall: 0.8919 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5644 - recall: 0.8693 - val_loss: 0.2597 - val_recall: 0.8784 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4908 - recall: 0.8738 - val_loss: 0.2307 - val_recall: 0.8874 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4413 - recall: 0.8994 - val_loss: 0.2294 - val_recall: 0.8874 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4386 - recall: 0.8874 - val_loss: 0.2230 - val_recall: 0.8964 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4308 - recall: 0.8996 - val_loss: 0.2273 - val_recall: 0.8964 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3995 - recall: 0.8941 - val_loss: 0.2143 - val_recall: 0.8964 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4082 - recall: 0.9002 - val_loss: 0.1864 - val_recall: 0.8964 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4184 - recall: 0.8849 - val_loss: 0.1963 - val_recall: 0.8919 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3834 - recall: 0.8952 - val_loss: 0.1848 - val_recall: 0.8964 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3765 - recall: 0.9043 - val_loss: 0.1827 - val_recall: 0.8874 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3778 - recall: 0.8941 - val_loss: 0.1703 - val_recall: 0.8964 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3544 - recall: 0.9012 - val_loss: 0.1776 - val_recall: 0.8919 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3466 - recall: 0.9055 - val_loss: 0.1565 - val_recall: 0.8919 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3444 - recall: 0.9066 - val_loss: 0.1756 - val_recall: 0.8919 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3449 - recall: 0.9026 - val_loss: 0.1720 - val_recall: 0.8874 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3511 - recall: 0.9044 - val_loss: 0.1699 - val_recall: 0.8874 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3444 - recall: 0.9024 - val_loss: 0.1694 - val_recall: 0.8874 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3283 - recall: 0.9088 - val_loss: 0.1961 - val_recall: 0.8874 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3358 - recall: 0.9073 - val_loss: 0.1749 - val_recall: 0.8919 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3412 - recall: 0.8935 - val_loss: 0.1635 - val_recall: 0.8874 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3268 - recall: 0.9082 - val_loss: 0.1578 - val_recall: 0.8874 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3424 - recall: 0.9088 - val_loss: 0.1650 - val_recall: 0.8874 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3041 - recall: 0.9200 - val_loss: 0.1584 - val_recall: 0.8874 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3453 - recall: 0.8925 - val_loss: 0.1617 - val_recall: 0.8874 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3257 - recall: 0.9083 - val_loss: 0.1633 - val_recall: 0.8829 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3062 - recall: 0.9085 - val_loss: 0.1552 - val_recall: 0.8829 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3075 - recall: 0.9164 - val_loss: 0.1710 - val_recall: 0.8829 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3256 - recall: 0.9125 - val_loss: 0.1580 - val_recall: 0.8919
print("Time taken in seconds ",end-start)
Time taken in seconds 31.806586503982544
model_3.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout_1 (Dropout) │ (None, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 14,213 (55.52 KB)
Trainable params: 4,737 (18.50 KB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 9,476 (37.02 KB)
plot(history,"loss")
model_3_train_perf=model_performance_classification(model_3,X_train,y_train)
model_3_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.990688 | 0.990688 | 0.990692 | 0.99069 |
model_3_val_perf=model_performance_classification(model_3,X_val,y_val)
model_3_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.98925 | 0.98925 | 0.989148 | 0.989192 |
Both recall have improved to 99% though the loss plot functions are still non-smooth
Model 4¶
Regularize with Batch Normalization
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_4 = Sequential()
model_4.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_4.add(BatchNormalization())
model_4.add(Dense(32,activation="relu"))
model_4.add(BatchNormalization())
model_4.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam(learning_rate=0.0001)
model_4.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_4.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.6216 - recall: 0.6038 - val_loss: 0.6390 - val_recall: 0.9189 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.9238 - recall: 0.9173 - val_loss: 0.5276 - val_recall: 0.9189 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.7494 - recall: 0.9252 - val_loss: 0.4343 - val_recall: 0.9054 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6397 - recall: 0.9190 - val_loss: 0.3623 - val_recall: 0.9009 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5604 - recall: 0.9164 - val_loss: 0.3077 - val_recall: 0.8874 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 4ms/step - loss: 0.5020 - recall: 0.9153 - val_loss: 0.2650 - val_recall: 0.8829 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.4585 - recall: 0.9153 - val_loss: 0.2344 - val_recall: 0.8784 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4245 - recall: 0.9137 - val_loss: 0.2105 - val_recall: 0.8739 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3989 - recall: 0.9154 - val_loss: 0.1928 - val_recall: 0.8739 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3781 - recall: 0.9156 - val_loss: 0.1788 - val_recall: 0.8739 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3608 - recall: 0.9152 - val_loss: 0.1677 - val_recall: 0.8739 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3454 - recall: 0.9152 - val_loss: 0.1598 - val_recall: 0.8694 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3326 - recall: 0.9152 - val_loss: 0.1518 - val_recall: 0.8694 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3207 - recall: 0.9178 - val_loss: 0.1459 - val_recall: 0.8649 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3096 - recall: 0.9178 - val_loss: 0.1414 - val_recall: 0.8649 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.2999 - recall: 0.9200 - val_loss: 0.1369 - val_recall: 0.8694 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2898 - recall: 0.9204 - val_loss: 0.1318 - val_recall: 0.8694 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2779 - recall: 0.9232 - val_loss: 0.1291 - val_recall: 0.8649 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2681 - recall: 0.9241 - val_loss: 0.1261 - val_recall: 0.8649 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2599 - recall: 0.9283 - val_loss: 0.1232 - val_recall: 0.8649 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2525 - recall: 0.9283 - val_loss: 0.1206 - val_recall: 0.8649 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2442 - recall: 0.9281 - val_loss: 0.1185 - val_recall: 0.8649 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2372 - recall: 0.9283 - val_loss: 0.1159 - val_recall: 0.8649 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2309 - recall: 0.9330 - val_loss: 0.1144 - val_recall: 0.8649 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2244 - recall: 0.9343 - val_loss: 0.1124 - val_recall: 0.8649 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.2188 - recall: 0.9351 - val_loss: 0.1106 - val_recall: 0.8649 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2128 - recall: 0.9356 - val_loss: 0.1086 - val_recall: 0.8649 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2068 - recall: 0.9404 - val_loss: 0.1072 - val_recall: 0.8649 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2019 - recall: 0.9427 - val_loss: 0.1052 - val_recall: 0.8649 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1962 - recall: 0.9452 - val_loss: 0.1041 - val_recall: 0.8649
print("Time taken in seconds ",end-start)
Time taken in seconds 36.642983198165894
model_4.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization │ (None, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_1 │ (None, 32) │ 128 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 14,981 (58.52 KB)
Trainable params: 4,929 (19.25 KB)
Non-trainable params: 192 (768.00 B)
Optimizer params: 9,860 (38.52 KB)
plot(history,"loss")
model_4_train_perf=model_performance_classification(model_4,X_train,y_train)
model_4_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 723us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.98625 | 0.98625 | 0.986826 | 0.986475 |
model_4_val_perf=model_performance_classification(model_4,X_val,y_val)
model_4_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 926us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.98275 | 0.98275 | 0.983112 | 0.982911 |
Recall in both cases has reduced to 98%. The loss plot has smoothen and gap between train and validation significantly reduced.
Model 5¶
Using both Batchnormalization and dropout
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_5 = Sequential()
model_5.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_5.add(BatchNormalization())
model_5.add(Dropout(0.4))
model_5.add(Dense(32,activation="relu"))
model_5.add(BatchNormalization())
model_5.add(Dropout(0.2))
model_5.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam()
model_5.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_5.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.1307 - recall: 0.7977 - val_loss: 0.2494 - val_recall: 0.8784 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6416 - recall: 0.8748 - val_loss: 0.1924 - val_recall: 0.8739 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5165 - recall: 0.8909 - val_loss: 0.1566 - val_recall: 0.8874 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5173 - recall: 0.8844 - val_loss: 0.1511 - val_recall: 0.8874 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4706 - recall: 0.8970 - val_loss: 0.1374 - val_recall: 0.8829 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4742 - recall: 0.9045 - val_loss: 0.1340 - val_recall: 0.8829 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4519 - recall: 0.9019 - val_loss: 0.1296 - val_recall: 0.8829 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4544 - recall: 0.8974 - val_loss: 0.1223 - val_recall: 0.8829 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4184 - recall: 0.9075 - val_loss: 0.1156 - val_recall: 0.8829 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4317 - recall: 0.8989 - val_loss: 0.1194 - val_recall: 0.8919 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4011 - recall: 0.9074 - val_loss: 0.1189 - val_recall: 0.8874 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4134 - recall: 0.9116 - val_loss: 0.1160 - val_recall: 0.8784 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3797 - recall: 0.9150 - val_loss: 0.1171 - val_recall: 0.8874 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3886 - recall: 0.9014 - val_loss: 0.1130 - val_recall: 0.8964 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3728 - recall: 0.9109 - val_loss: 0.1111 - val_recall: 0.8874 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3694 - recall: 0.9051 - val_loss: 0.1073 - val_recall: 0.8874 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3878 - recall: 0.9084 - val_loss: 0.1064 - val_recall: 0.8874 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3711 - recall: 0.9168 - val_loss: 0.1062 - val_recall: 0.8874 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3599 - recall: 0.9109 - val_loss: 0.1052 - val_recall: 0.8829 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3674 - recall: 0.9099 - val_loss: 0.1010 - val_recall: 0.8829 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3794 - recall: 0.9046 - val_loss: 0.1014 - val_recall: 0.8829 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3483 - recall: 0.9132 - val_loss: 0.1043 - val_recall: 0.8919 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3609 - recall: 0.9163 - val_loss: 0.1026 - val_recall: 0.8829 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3464 - recall: 0.9084 - val_loss: 0.1021 - val_recall: 0.8874 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3621 - recall: 0.9148 - val_loss: 0.1082 - val_recall: 0.8964 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3311 - recall: 0.9143 - val_loss: 0.1051 - val_recall: 0.8919 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3281 - recall: 0.9236 - val_loss: 0.1040 - val_recall: 0.8919 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.3419 - recall: 0.9151 - val_loss: 0.0971 - val_recall: 0.8919 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3251 - recall: 0.9160 - val_loss: 0.1006 - val_recall: 0.8919 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3252 - recall: 0.9173 - val_loss: 0.0961 - val_recall: 0.8874
print("Time taken in seconds ",end-start)
Time taken in seconds 36.77445697784424
model_5.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization │ (None, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_1 │ (None, 32) │ 128 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout_1 (Dropout) │ (None, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 14,981 (58.52 KB)
Trainable params: 4,929 (19.25 KB)
Non-trainable params: 192 (768.00 B)
Optimizer params: 9,860 (38.52 KB)
plot(history,"loss")
model_5_train_perf=model_performance_classification(model_5,X_train,y_train)
model_5_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 993us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.989375 | 0.989375 | 0.989473 | 0.98942 |
model_5_val_perf=model_performance_classification(model_5,X_val,y_val)
model_5_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.9875 | 0.9875 | 0.9875 | 0.9875 |
Both perfomance in regard to recall are at 98%. The loss plots are non-smooth
Model 6¶
Using weight initialization and dropout
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_6 = Sequential()
model_6.add(Dense(64,activation="relu",input_dim = X_train.shape[1],kernel_initializer='he_normal'))
model_6.add(Dropout(0.4))
model_6.add(Dense(32,activation="relu",kernel_initializer='he_normal'))
model_6.add(Dropout(0.2))
model_6.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam()
model_6.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_6.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.9393 - recall: 0.6566 - val_loss: 0.3760 - val_recall: 0.8829 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.8441 - recall: 0.8062 - val_loss: 0.3082 - val_recall: 0.8874 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6197 - recall: 0.8560 - val_loss: 0.2725 - val_recall: 0.8829 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5885 - recall: 0.8529 - val_loss: 0.2644 - val_recall: 0.8919 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5384 - recall: 0.8843 - val_loss: 0.2571 - val_recall: 0.8784 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4891 - recall: 0.8885 - val_loss: 0.2639 - val_recall: 0.8784 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4619 - recall: 0.8862 - val_loss: 0.2133 - val_recall: 0.8784 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.4493 - recall: 0.8742 - val_loss: 0.2134 - val_recall: 0.8874 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4468 - recall: 0.8824 - val_loss: 0.2146 - val_recall: 0.8919 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4250 - recall: 0.8874 - val_loss: 0.2157 - val_recall: 0.8829 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4205 - recall: 0.8945 - val_loss: 0.1953 - val_recall: 0.8829 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3911 - recall: 0.9025 - val_loss: 0.1974 - val_recall: 0.8919 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3925 - recall: 0.8878 - val_loss: 0.1863 - val_recall: 0.8874 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4054 - recall: 0.8866 - val_loss: 0.1832 - val_recall: 0.9009 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3833 - recall: 0.8860 - val_loss: 0.1866 - val_recall: 0.9009 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3544 - recall: 0.9095 - val_loss: 0.1765 - val_recall: 0.8964 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3699 - recall: 0.8974 - val_loss: 0.1845 - val_recall: 0.8964 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3412 - recall: 0.9120 - val_loss: 0.1730 - val_recall: 0.8919 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3595 - recall: 0.9030 - val_loss: 0.1667 - val_recall: 0.8874 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3499 - recall: 0.9066 - val_loss: 0.1670 - val_recall: 0.9009 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3585 - recall: 0.9036 - val_loss: 0.1723 - val_recall: 0.8919 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3324 - recall: 0.9080 - val_loss: 0.1634 - val_recall: 0.8874 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3364 - recall: 0.9035 - val_loss: 0.1696 - val_recall: 0.8964 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3326 - recall: 0.9075 - val_loss: 0.1801 - val_recall: 0.9009 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3219 - recall: 0.9076 - val_loss: 0.1701 - val_recall: 0.8919 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3239 - recall: 0.8982 - val_loss: 0.1535 - val_recall: 0.8964 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3150 - recall: 0.9148 - val_loss: 0.1679 - val_recall: 0.8919 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3318 - recall: 0.9097 - val_loss: 0.1603 - val_recall: 0.8874 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3095 - recall: 0.9121 - val_loss: 0.1662 - val_recall: 0.8919 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3266 - recall: 0.9113 - val_loss: 0.1726 - val_recall: 0.8964
print("Time taken in seconds ",end-start)
Time taken in seconds 35.58531212806702
model_6.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout_1 (Dropout) │ (None, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 14,213 (55.52 KB)
Trainable params: 4,737 (18.50 KB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 9,476 (37.02 KB)
plot(history,"loss")
model_6_train_perf=model_performance_classification(model_6,X_train,y_train)
model_6_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 682us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.990938 | 0.990938 | 0.990923 | 0.99093 |
model_6_val_perf=model_performance_classification(model_6,X_val,y_val)
model_6_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.9885 | 0.9885 | 0.9885 | 0.9885 |
Both recals are around 99% and loss plots are smoothening
Model 7¶
Weight initialization and Batch Normalization
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_7 = Sequential()
model_7.add(Dense(64,activation="relu",input_dim = X_train.shape[1],kernel_initializer='he_normal'))
model_7.add(BatchNormalization())
model_7.add(Dense(32,activation="relu",kernel_initializer='he_normal'))
model_7.add(BatchNormalization())
model_7.add(Dense(1,activation = 'sigmoid'))
#compiling the model with Adam as optimiser
optimizer = keras.optimizers.Adam()
model_7.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_7.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 4s 4ms/step - loss: 0.8269 - recall: 0.8495 - val_loss: 0.2436 - val_recall: 0.8874 Epoch 2/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 3s 4ms/step - loss: 0.4502 - recall: 0.9085 - val_loss: 0.1806 - val_recall: 0.8874 Epoch 3/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3810 - recall: 0.9101 - val_loss: 0.1518 - val_recall: 0.8919 Epoch 4/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3420 - recall: 0.9193 - val_loss: 0.1358 - val_recall: 0.8829 Epoch 5/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3069 - recall: 0.9251 - val_loss: 0.1351 - val_recall: 0.8919 Epoch 6/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2787 - recall: 0.9344 - val_loss: 0.1294 - val_recall: 0.8919 Epoch 7/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2627 - recall: 0.9336 - val_loss: 0.1273 - val_recall: 0.8919 Epoch 8/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2485 - recall: 0.9327 - val_loss: 0.1219 - val_recall: 0.8964 Epoch 9/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2287 - recall: 0.9410 - val_loss: 0.1196 - val_recall: 0.8919 Epoch 10/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2274 - recall: 0.9392 - val_loss: 0.1166 - val_recall: 0.8919 Epoch 11/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2068 - recall: 0.9497 - val_loss: 0.1122 - val_recall: 0.8874 Epoch 12/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1832 - recall: 0.9531 - val_loss: 0.1144 - val_recall: 0.8874 Epoch 13/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1739 - recall: 0.9573 - val_loss: 0.1041 - val_recall: 0.8874 Epoch 14/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1623 - recall: 0.9588 - val_loss: 0.1029 - val_recall: 0.8694 Epoch 15/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1551 - recall: 0.9604 - val_loss: 0.0987 - val_recall: 0.8739 Epoch 16/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1395 - recall: 0.9626 - val_loss: 0.1025 - val_recall: 0.8874 Epoch 17/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1260 - recall: 0.9639 - val_loss: 0.0908 - val_recall: 0.8694 Epoch 18/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1334 - recall: 0.9629 - val_loss: 0.0908 - val_recall: 0.8694 Epoch 19/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1130 - recall: 0.9697 - val_loss: 0.1303 - val_recall: 0.8784 Epoch 20/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1184 - recall: 0.9680 - val_loss: 0.0834 - val_recall: 0.8694 Epoch 21/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0891 - recall: 0.9777 - val_loss: 0.0903 - val_recall: 0.8649 Epoch 22/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.0900 - recall: 0.9766 - val_loss: 0.0883 - val_recall: 0.8649 Epoch 23/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0859 - recall: 0.9758 - val_loss: 0.0891 - val_recall: 0.8694 Epoch 24/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0808 - recall: 0.9809 - val_loss: 0.0867 - val_recall: 0.8423 Epoch 25/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0684 - recall: 0.9847 - val_loss: 0.0953 - val_recall: 0.8649 Epoch 26/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0555 - recall: 0.9896 - val_loss: 0.0862 - val_recall: 0.8514 Epoch 27/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0625 - recall: 0.9825 - val_loss: 0.1080 - val_recall: 0.8468 Epoch 28/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0870 - recall: 0.9816 - val_loss: 0.0921 - val_recall: 0.8694 Epoch 29/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0701 - recall: 0.9832 - val_loss: 0.0928 - val_recall: 0.8739 Epoch 30/30 500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0501 - recall: 0.9902 - val_loss: 0.0931 - val_recall: 0.8694
print("Time taken in seconds ",end-start)
Time taken in seconds 38.600029945373535
model_7.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 2,624 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization │ (None, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_1 │ (None, 32) │ 128 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 14,981 (58.52 KB)
Trainable params: 4,929 (19.25 KB)
Non-trainable params: 192 (768.00 B)
Optimizer params: 9,860 (38.52 KB)
plot(history,"loss")
#training performance
model_7_train_perf=model_performance_classification(model_7,X_train,y_train)
model_7_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 785us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.986812 | 0.986812 | 0.987221 | 0.986979 |
#validation performance
model_7_val_perf=model_performance_classification(model_7,X_val,y_val)
model_7_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 902us/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.98425 | 0.98425 | 0.984428 | 0.984332 |
Model Performance Comparison and Final Model Selection¶
Now, in order to select the final model, we will compare the performances of all the models for the training and validation sets.
#training performance comparision
model_train_perf_comp=pd.concat([model_0_train_perf.T,model_1_train_perf.T,model_2_train_perf.T,
model_3_train_perf.T,model_4_train_perf.T, model_5_train_perf.T,
model_6_train_perf.T,model_7_train_perf.T,
],
axis=1
)
model_train_perf_comp.columns=[
"Neural Network (SGD, No Regularization)",
"Neural Network (SGD with Momentum, No Regularization)",
"Neural Network (Adam , No Regularization)",
"Neural Network (Adam, dropout [0.4,0.2])",
"Neural Network (Adam, Batch Normalization)",
"Neural Network (Adam,dropout [0.4,0.2], Batch Normalization)",
"Neural Network (Adam,dropout [0.4,0.2] ,He initialization)",
"Neural Network (Adam,He initialization, Batch Normalization)"
]
print("Training perfomance comparision:")
model_train_perf_comp
Training perfomance comparision:
| Neural Network (SGD, No Regularization) | Neural Network (SGD with Momentum, No Regularization) | Neural Network (Adam , No Regularization) | Neural Network (Adam, dropout [0.4,0.2]) | Neural Network (Adam, Batch Normalization) | Neural Network (Adam,dropout [0.4,0.2], Batch Normalization) | Neural Network (Adam,dropout [0.4,0.2] ,He initialization) | Neural Network (Adam,He initialization, Batch Normalization) | |
|---|---|---|---|---|---|---|---|---|
| Accuracy | 0.981750 | 0.993437 | 0.987812 | 0.990688 | 0.986250 | 0.989375 | 0.990938 | 0.986812 |
| Recall | 0.981750 | 0.993437 | 0.987812 | 0.990688 | 0.986250 | 0.989375 | 0.990938 | 0.986812 |
| Precision | 0.984608 | 0.993372 | 0.989453 | 0.990692 | 0.986826 | 0.989473 | 0.990923 | 0.987221 |
| F1 Score | 0.982628 | 0.993393 | 0.988276 | 0.990690 | 0.986475 | 0.989420 | 0.990930 | 0.986979 |
#validation perfomance comparision
model_val_perf_comp=pd.concat([model_0_val_perf.T,model_1_val_perf.T,model_2_val_perf.T, model_3_val_perf.T,
model_4_val_perf.T,model_5_val_perf.T,model_6_val_perf.T,model_7_val_perf.T,
],axis=1
)
model_val_perf_comp.columns=[
"Neural Network (SGD, No Regularization)",
"Neural Network (SGD with Momentum, No Regularization)",
"Neural Network (Adam , No Regularization)",
"Neural Network (Adam, dropout [0.4,0.2])",
"Neural Network (Adam, Batch Normalization)",
"Neural Network (Adam,dropout [0.4,0.2], Batch Normalization)",
"Neural Network (Adam,dropout [0.4,0.2] ,He initialization)",
"Neural Network (Adam,He initialization, Batch Normalization)"
]
print("Validation perfomance comparision:")
model_val_perf_comp
Validation perfomance comparision:
| Neural Network (SGD, No Regularization) | Neural Network (SGD with Momentum, No Regularization) | Neural Network (Adam , No Regularization) | Neural Network (Adam, dropout [0.4,0.2]) | Neural Network (Adam, Batch Normalization) | Neural Network (Adam,dropout [0.4,0.2], Batch Normalization) | Neural Network (Adam,dropout [0.4,0.2] ,He initialization) | Neural Network (Adam,He initialization, Batch Normalization) | |
|---|---|---|---|---|---|---|---|---|
| Accuracy | 0.978000 | 0.988000 | 0.979250 | 0.989250 | 0.982750 | 0.9875 | 0.9885 | 0.984250 |
| Recall | 0.978000 | 0.988000 | 0.979250 | 0.989250 | 0.982750 | 0.9875 | 0.9885 | 0.984250 |
| Precision | 0.980238 | 0.987951 | 0.980815 | 0.989148 | 0.983112 | 0.9875 | 0.9885 | 0.984428 |
| F1 Score | 0.978812 | 0.987974 | 0.979846 | 0.989192 | 0.982911 | 0.9875 | 0.9885 | 0.984332 |
Neural network model 3 with Adam as optimizer and dropout[0.4,0.2] as regularization is chosen since it has the highest recall hence best in addressing false negative associated with high replacement cost
Now, let's check the performance of the final model on the test set.
#Perfomance of model_3 on test
model_3_test_perf=model_performance_classification(model_3,X_test,y_test)
model_3_test_perf
157/157 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step
| Accuracy | Recall | Precision | F1 Score | |
|---|---|---|---|---|
| 0 | 0.9882 | 0.9882 | 0.987997 | 0.988069 |
The perfomance is within range with a recall of 98.82 percent, suggesting the model is accurately classifying instances with minimal false negatives.
Actionable Insights and Recommendations¶
We have build a model that can be used to predict failures so that the generators could be repaired before failing/breaking to reduce the overall maintenance cost.
The model is accurately classifying instances with minimal false negatives which has the highest maintenance cost interms of replacement cost hence gradually reducing maintanance cost.
The company can deploy the final model from this exercise to identify with a reasonable degree of accuracy whether failures will occur or not, and this process seems to be easier and more time-efficient than other methods
The model could further be continously trained on large data to enhance it performance.